--- title: Dataloaders keywords: fastai sidebar: home_sidebar summary: "Loading data for training DL models." description: "Loading data for training DL models." nb_path: "nbs/04_dataloaders.ipynb" ---
{% raw %}
{% endraw %} {% raw %}
{% endraw %}

Fast.ai DataBlock API

In this module you can have functions for a classification and a segmentation problem.

With the DataBlock API of fast.ai we can build in a easy way to training (with train attribute) and valid (with valid attribute) the datasets inside a Datasets fastai class.

With a DataLoaders we can load the data into a GPU applying the transforms. It calls the Pytorch DataLoader on each subset of Datasets.

The list of tfms applied is:

  • after_item: applied on each item after grabbing it inside the dataset.

  • before_batch: applied on the list of items before they are collated.

  • after_batch: applied on the batch as a whole after its construction.

Classification

{% raw %} 
b_tfms = aug_transforms(
            size=size if size else (256, 1600),
            max_warp=.1,
            max_rotate=5.,
            max_lighting=0.1)
{% endraw %} {% raw %} 
b_tfms

[Flip -- {'size': (256, 1600), 'mode': 'bilinear', 'pad_mode': 'reflection', 'mode_mask': 'nearest', 'align_corners': True, 'p': 0.5}:
 encodes: (TensorImage,object) -> encodes
 (TensorMask,object) -> encodes
 (TensorBBox,object) -> encodes
 (TensorPoint,object) -> encodes
 decodes: ,
 Brightness -- {'max_lighting': 0.1, 'p': 1.0, 'draw': None, 'batch': False}:
 encodes: (TensorImage,object) -> encodes
 decodes: ]
{% endraw %} {% raw %} 
dblock = DataBlock(
    blocks=(ImageBlock, MultiCategoryBlock()),
    get_x=ColReader(0, pref=train_path),
    get_y=ColReader(1, label_delim=' '),
    splitter=RandomSplitter(valid_pct=0.2, seed=42),
    batch_tfms=b_tfms)
{% endraw %}

Now we can check that all is alright passing a source into the dblock and the API returns all the process explained.

{% raw %} 
dblock.summary(train_multi, show_batch=True)

Setting-up type transforms pipelines
Collecting items from ClassId        ImageId ClassId_multi
0        0002cc93b.jpg             1
1        00031f466.jpg             0
2        000418bfc.jpg             0
3        000789191.jpg             0
4        0007a71bf.jpg             3
...                ...           ...
12563    fff0295e1.jpg             0
12564    fff02e9c5.jpg             3
12565    fffe98443.jpg             3
12566    ffff4eaa8.jpg             3
12567    ffffd67df.jpg             3

[12568 rows x 2 columns]
Found 12568 items
2 datasets of sizes 10055,2513
Setting up Pipeline: ColReader -- {'cols': 0, 'pref': Path('../data/train_images'), 'suff': '', 'label_delim': None} -> PILBase.create
Setting up Pipeline: ColReader -- {'cols': 1, 'pref': '', 'suff': '', 'label_delim': ' '} -> MultiCategorize -- {'vocab': None, 'sort': True, 'add_na': False} -> OneHotEncode -- {'c': None}

Building one sample
  Pipeline: ColReader -- {'cols': 0, 'pref': Path('../data/train_images'), 'suff': '', 'label_delim': None} -> PILBase.create
    starting from
      ClassId
ImageId          0fac62a3e.jpg
ClassId_multi                0
Name: 761, dtype: object
    applying ColReader -- {'cols': 0, 'pref': Path('../data/train_images'), 'suff': '', 'label_delim': None} gives
      ../data/train_images/0fac62a3e.jpg
    applying PILBase.create gives
      PILImage mode=RGB size=1600x256
  Pipeline: ColReader -- {'cols': 1, 'pref': '', 'suff': '', 'label_delim': ' '} -> MultiCategorize -- {'vocab': None, 'sort': True, 'add_na': False} -> OneHotEncode -- {'c': None}
    starting from
      ClassId
ImageId          0fac62a3e.jpg
ClassId_multi                0
Name: 761, dtype: object
    applying ColReader -- {'cols': 1, 'pref': '', 'suff': '', 'label_delim': ' '} gives
      [0]
    applying MultiCategorize -- {'vocab': None, 'sort': True, 'add_na': False} gives
      TensorMultiCategory([0])
    applying OneHotEncode -- {'c': None} gives
      TensorMultiCategory([1., 0., 0., 0., 0.])

Final sample: (PILImage mode=RGB size=1600x256, TensorMultiCategory([1., 0., 0., 0., 0.]))


Collecting items from ClassId        ImageId ClassId_multi
0        0002cc93b.jpg             1
1        00031f466.jpg             0
2        000418bfc.jpg             0
3        000789191.jpg             0
4        0007a71bf.jpg             3
...                ...           ...
12563    fff0295e1.jpg             0
12564    fff02e9c5.jpg             3
12565    fffe98443.jpg             3
12566    ffff4eaa8.jpg             3
12567    ffffd67df.jpg             3

[12568 rows x 2 columns]
Found 12568 items
2 datasets of sizes 10055,2513
Setting up Pipeline: ColReader -- {'cols': 0, 'pref': Path('../data/train_images'), 'suff': '', 'label_delim': None} -> PILBase.create
Setting up Pipeline: ColReader -- {'cols': 1, 'pref': '', 'suff': '', 'label_delim': ' '} -> MultiCategorize -- {'vocab': None, 'sort': True, 'add_na': False} -> OneHotEncode -- {'c': None}
Setting up after_item: Pipeline: ToTensor
Setting up before_batch: Pipeline: 
Setting up after_batch: Pipeline: IntToFloatTensor -- {'div': 255.0, 'div_mask': 1} -> Flip -- {'size': (256, 1600), 'mode': 'bilinear', 'pad_mode': 'reflection', 'mode_mask': 'nearest', 'align_corners': True, 'p': 0.5} -> Brightness -- {'max_lighting': 0.1, 'p': 1.0, 'draw': None, 'batch': False}

Building one batch
Applying item_tfms to the first sample:
  Pipeline: ToTensor
    starting from
      (PILImage mode=RGB size=1600x256, TensorMultiCategory([1., 0., 0., 0., 0.]))
    applying ToTensor gives
      (TensorImage of size 3x256x1600, TensorMultiCategory([1., 0., 0., 0., 0.]))

Adding the next 3 samples

No before_batch transform to apply

Collating items in a batch

Applying batch_tfms to the batch built
  Pipeline: IntToFloatTensor -- {'div': 255.0, 'div_mask': 1} -> Flip -- {'size': (256, 1600), 'mode': 'bilinear', 'pad_mode': 'reflection', 'mode_mask': 'nearest', 'align_corners': True, 'p': 0.5} -> Brightness -- {'max_lighting': 0.1, 'p': 1.0, 'draw': None, 'batch': False}
    starting from
      (TensorImage of size 4x3x256x1600, TensorMultiCategory of size 4x5)
    applying IntToFloatTensor -- {'div': 255.0, 'div_mask': 1} gives
      (TensorImage of size 4x3x256x1600, TensorMultiCategory of size 4x5)
    applying Flip -- {'size': (256, 1600), 'mode': 'bilinear', 'pad_mode': 'reflection', 'mode_mask': 'nearest', 'align_corners': True, 'p': 0.5} gives
      (TensorImage of size 4x3x256x1600, TensorMultiCategory of size 4x5)
    applying Brightness -- {'max_lighting': 0.1, 'p': 1.0, 'draw': None, 'batch': False} gives
      (TensorImage of size 4x3x256x1600, TensorMultiCategory of size 4x5)
{% endraw %}

To create DataLoaders the batch size is needed.

{% raw %} 
dloader = dblock.dataloaders(train_multi, bs=8, verbose=True)
type(dloader)

Collecting items from ClassId        ImageId ClassId_multi
0        0002cc93b.jpg             1
1        00031f466.jpg             0
2        000418bfc.jpg             0
3        000789191.jpg             0
4        0007a71bf.jpg             3
...                ...           ...
12563    fff0295e1.jpg             0
12564    fff02e9c5.jpg             3
12565    fffe98443.jpg             3
12566    ffff4eaa8.jpg             3
12567    ffffd67df.jpg             3

[12568 rows x 2 columns]
Found 12568 items
2 datasets of sizes 10055,2513
Setting up Pipeline: ColReader -- {'cols': 0, 'pref': Path('../data/train_images'), 'suff': '', 'label_delim': None} -> PILBase.create
Setting up Pipeline: ColReader -- {'cols': 1, 'pref': '', 'suff': '', 'label_delim': ' '} -> MultiCategorize -- {'vocab': None, 'sort': True, 'add_na': False} -> OneHotEncode -- {'c': None}
Setting up after_item: Pipeline: ToTensor
Setting up before_batch: Pipeline: 
Setting up after_batch: Pipeline: IntToFloatTensor -- {'div': 255.0, 'div_mask': 1} -> Flip -- {'size': (256, 1600), 'mode': 'bilinear', 'pad_mode': 'reflection', 'mode_mask': 'nearest', 'align_corners': True, 'p': 0.5} -> Brightness -- {'max_lighting': 0.1, 'p': 1.0, 'draw': None, 'batch': False}

fastai.data.core.DataLoaders
{% endraw %}

An example of pipeline:

{% raw %} 
f = dloader.after_item
items = [f(dloader.create_item(i)) for i in range(4)]
items[0][0].shape, items[0][1]

((3, 256, 1600), TensorMultiCategory([1., 0., 0., 0., 0.]))
{% endraw %} {% raw %} 
batch = dloader.do_batch([item for item in items]) # apply before_batch
batch[0].shape, batch[1]

((4, 3, 256, 1600),
 TensorMultiCategory([[1., 0., 0., 0., 0.],
         [1., 0., 0., 0., 0.],
         [0., 0., 0., 1., 0.],
         [1., 0., 0., 0., 0.]]))
{% endraw %} {% raw %} 
show_image_batch(batch, items=4, cols=2, figsize=(15,5))
{% endraw %} {% raw %} 
batch_tfms = dloader.after_batch(batch)
{% endraw %} {% raw %} 
show_image_batch(batch_tfms, items=4, cols=2, figsize=(15,5))
{% endraw %} {% raw %}
{% endraw %} {% raw %}

get_classification_dls[source]

get_classification_dls(bs, with_tfms:bool=True, size=None, seed=42)

Dataloaders from train DataFrame

{% endraw %} {% raw %} 
bs = 4
dls = get_classification_dls(bs)
{% endraw %} {% raw %} 
dls.train.show_batch(figsize=(15, 3))
{% endraw %} {% raw %} 
dls.valid.show_batch(figsize=(15, 3))
{% endraw %} {% raw %} 
x, y = dls.train.one_batch()
x.shape, y.shape

((4, 3, 256, 1600), (4, 5))
{% endraw %}

Segmentation

To get a DataLoaders object for training segmentation FastAI models, we need to:

  • load the training images
  • load the corrisponding labels (saved in labels_dir by the preprocessing module with the create_mask function)
  • pair the images with labels with a Dataset
  • split the dataset into training and validation and pair them with a Datasets object.
  • for each dataset, create a DataLoader with a batch size and pair them with a DataLoaders object.
{% raw %}
{% endraw %} {% raw %} 
btfms = aug_transforms(
            size=size, 
            max_warp=.1,
            max_rotate=5.,
            max_lighting=0.1)
btfms

[Flip -- {'size': (256, 1600), 'mode': 'bilinear', 'pad_mode': 'reflection', 'mode_mask': 'nearest', 'align_corners': True, 'p': 0.5}:
 encodes: (TensorImage,object) -> encodes
 (TensorMask,object) -> encodes
 (TensorBBox,object) -> encodes
 (TensorPoint,object) -> encodes
 decodes: ,
 Brightness -- {'max_lighting': 0.1, 'p': 1.0, 'draw': None, 'batch': False}:
 encodes: (TensorImage,object) -> encodes
 decodes: ]
{% endraw %} {% raw %} 
def get_x(s):
    img_name = s["ImageId"]
    return train_path / str(img_name)

def get_y(s):
    img_name = s["ImageId"].split(".")[0] + "_P.png"
    return labels_path / img_name

dblock = DataBlock(
    blocks=(ImageBlock, MaskBlock(codes=classes)),
    get_x=get_x,
    get_y=get_y,
    splitter=RandomSplitter(valid_pct=0.2, seed=42),
    batch_tfms=btfms)
{% endraw %} {% raw %} 
dblock.summary(train_multi, show_batch=True)

Setting-up type transforms pipelines
Collecting items from ClassId        ImageId ClassId_multi
0        0002cc93b.jpg             1
1        00031f466.jpg             0
2        000418bfc.jpg             0
3        000789191.jpg             0
4        0007a71bf.jpg             3
...                ...           ...
12563    fff0295e1.jpg             0
12564    fff02e9c5.jpg             3
12565    fffe98443.jpg             3
12566    ffff4eaa8.jpg             3
12567    ffffd67df.jpg             3

[12568 rows x 2 columns]
Found 12568 items
2 datasets of sizes 10055,2513
Setting up Pipeline: get_x -> PILBase.create
Setting up Pipeline: get_y -> PILBase.create

Building one sample
  Pipeline: get_x -> PILBase.create
    starting from
      ClassId
ImageId          0fac62a3e.jpg
ClassId_multi                0
Name: 761, dtype: object
    applying get_x gives
      ../data/train_images/0fac62a3e.jpg
    applying PILBase.create gives
      PILImage mode=RGB size=1600x256
  Pipeline: get_y -> PILBase.create
    starting from
      ClassId
ImageId          0fac62a3e.jpg
ClassId_multi                0
Name: 761, dtype: object
    applying get_y gives
      ../data/labels/0fac62a3e_P.png
    applying PILBase.create gives
      PILMask mode=L size=1600x256

Final sample: (PILImage mode=RGB size=1600x256, PILMask mode=L size=1600x256)


Collecting items from ClassId        ImageId ClassId_multi
0        0002cc93b.jpg             1
1        00031f466.jpg             0
2        000418bfc.jpg             0
3        000789191.jpg             0
4        0007a71bf.jpg             3
...                ...           ...
12563    fff0295e1.jpg             0
12564    fff02e9c5.jpg             3
12565    fffe98443.jpg             3
12566    ffff4eaa8.jpg             3
12567    ffffd67df.jpg             3

[12568 rows x 2 columns]
Found 12568 items
2 datasets of sizes 10055,2513
Setting up Pipeline: get_x -> PILBase.create
Setting up Pipeline: get_y -> PILBase.create
Setting up after_item: Pipeline: AddMaskCodes -> ToTensor
Setting up before_batch: Pipeline: 
Setting up after_batch: Pipeline: IntToFloatTensor -- {'div': 255.0, 'div_mask': 1} -> Flip -- {'size': (256, 1600), 'mode': 'bilinear', 'pad_mode': 'reflection', 'mode_mask': 'nearest', 'align_corners': True, 'p': 0.5} -> Brightness -- {'max_lighting': 0.1, 'p': 1.0, 'draw': None, 'batch': False}

Building one batch
Applying item_tfms to the first sample:
  Pipeline: AddMaskCodes -> ToTensor
    starting from
      (PILImage mode=RGB size=1600x256, PILMask mode=L size=1600x256)
    applying AddMaskCodes gives
      (PILImage mode=RGB size=1600x256, PILMask mode=L size=1600x256)
    applying ToTensor gives
      (TensorImage of size 3x256x1600, TensorMask of size 256x1600)

Adding the next 3 samples

No before_batch transform to apply

Collating items in a batch

Applying batch_tfms to the batch built
  Pipeline: IntToFloatTensor -- {'div': 255.0, 'div_mask': 1} -> Flip -- {'size': (256, 1600), 'mode': 'bilinear', 'pad_mode': 'reflection', 'mode_mask': 'nearest', 'align_corners': True, 'p': 0.5} -> Brightness -- {'max_lighting': 0.1, 'p': 1.0, 'draw': None, 'batch': False}
    starting from
      (TensorImage of size 4x3x256x1600, TensorMask of size 4x256x1600)
    applying IntToFloatTensor -- {'div': 255.0, 'div_mask': 1} gives
      (TensorImage of size 4x3x256x1600, TensorMask of size 4x256x1600)
    applying Flip -- {'size': (256, 1600), 'mode': 'bilinear', 'pad_mode': 'reflection', 'mode_mask': 'nearest', 'align_corners': True, 'p': 0.5} gives
      (TensorImage of size 4x3x256x1600, TensorMask of size 4x256x1600)
    applying Brightness -- {'max_lighting': 0.1, 'p': 1.0, 'draw': None, 'batch': False} gives
      (TensorImage of size 4x3x256x1600, TensorMask of size 4x256x1600)
{% endraw %} {% raw %} 
dloader = dblock.dataloaders(train_multi, verbose=True, bs=4)

Collecting items from ClassId        ImageId ClassId_multi
0        0002cc93b.jpg             1
1        00031f466.jpg             0
2        000418bfc.jpg             0
3        000789191.jpg             0
4        0007a71bf.jpg             3
...                ...           ...
12563    fff0295e1.jpg             0
12564    fff02e9c5.jpg             3
12565    fffe98443.jpg             3
12566    ffff4eaa8.jpg             3
12567    ffffd67df.jpg             3

[12568 rows x 2 columns]
Found 12568 items
2 datasets of sizes 10055,2513
Setting up Pipeline: get_x -> PILBase.create
Setting up Pipeline: get_y -> PILBase.create
Setting up after_item: Pipeline: AddMaskCodes -> ToTensor
Setting up before_batch: Pipeline: 
Setting up after_batch: Pipeline: IntToFloatTensor -- {'div': 255.0, 'div_mask': 1} -> Flip -- {'size': (256, 1600), 'mode': 'bilinear', 'pad_mode': 'reflection', 'mode_mask': 'nearest', 'align_corners': True, 'p': 0.5} -> Brightness -- {'max_lighting': 0.1, 'p': 1.0, 'draw': None, 'batch': False}
{% endraw %} {% raw %} 
type(dloader)

fastai.data.core.DataLoaders
{% endraw %} {% raw %} 
dloader.train_ds[0]

(PILImage mode=RGB size=1600x256, PILMask mode=L size=1600x256)
{% endraw %} {% raw %} 
xb, yb = next(iter(dloader.train))
xb.shape, yb.shape
{% endraw %}

An example of pipeline:

{% raw %} 
elems = [13, 17, 25]
n = len(elems)
figszs = (40,20)
imgszs = (25,5)
{% endraw %} {% raw %} 
f = dloader.after_item
items = [f(dloader.create_item(i)) for i in elems]
items[0][0].shape, items[0][1].shape, items[0][0].max(), items[0][0].min()

((3, 256, 1600),
 (256, 1600),
 TensorImage(255, dtype=torch.uint8),
 TensorImage(4, dtype=torch.uint8))
{% endraw %} {% raw %} 
img_batch, mask_batch = dloader.do_batch([item for item in items])
img_batch.shape, mask_batch.shape

((3, 3, 256, 1600), (3, 256, 1600))
{% endraw %} {% raw %} 
fig, axs = plt.subplots(n, 1, figsize=figszs)
for i in range(len(axs)):
    show_image(img_batch[i], ax=axs[i], figsize=imgszs)
{% endraw %} {% raw %} 
fig, axs = plt.subplots(n, 1, figsize=figszs)
for i in range(len(axs)):
    show_image(mask_batch[i], ax=axs[i], figsize=imgszs)
{% endraw %} {% raw %} 
img_tfbatch, mask_tfbatch = dloader.after_batch((img_batch, mask_batch))
img_tfbatch.shape, mask_tfbatch.shape 

((3, 3, 256, 1600), (3, 256, 1600))
{% endraw %} {% raw %} 
fig, axs = plt.subplots(n, 1, figsize=figszs)
for i in range(len(axs)):
    show_image(img_tfbatch[i], ax=axs[i], figsize=imgszs)
{% endraw %} {% raw %} 
fig, axs = plt.subplots(n, 1, figsize=figszs)
for i in range(len(axs)):
    show_image(mask_tfbatch[i], ax=axs[i], figsize=imgszs)
{% endraw %} {% raw %} 
for img in img_tfbatch:
    print(img.min(), img.max())

tensor(0.0426) tensor(1.0000)
tensor(0.0004) tensor(1.0000)
tensor(0.1151) tensor(1.0000)
{% endraw %} {% raw %} 
for mask in mask_batch:
    print(mask.min(), mask.max())

tensor(0, dtype=torch.uint8) tensor(4, dtype=torch.uint8)
tensor(0, dtype=torch.uint8) tensor(3, dtype=torch.uint8)
tensor(0, dtype=torch.uint8) tensor(3, dtype=torch.uint8)
{% endraw %}

The get_segmentation_dls will load from the folder all the images while the get_segmentation_dls_from_df loads the images from a custom DataFrame to train on a different subsample.

{% raw %}
{% endraw %} {% raw %}

get_segmentation_dls[source]

get_segmentation_dls(bs, size, with_btfms=True, seed=42)

Dataloaders from train_path folder

{% endraw %} {% raw %}
{% endraw %} {% raw %}

get_segmentation_dls_from_df[source]

get_segmentation_dls_from_df(train_df, bs, size, with_btfms=True, seed=42)

Dataloaders from train DataFrame

{% endraw %} {% raw %} 
bs = 4
szs = (128, 800)
dls = get_segmentation_dls_from_df(train_multi, bs, szs)
{% endraw %} {% raw %} 
dls.train.show_batch(figsize=(15, 3))
{% endraw %} {% raw %} 
dls.valid.show_batch(figsize=(15, 3))
{% endraw %} {% raw %} 
x, y = dls.train.one_batch()
x.shape, y.shape

((4, 3, 128, 800), (4, 128, 800))
{% endraw %} {% raw %} 
[torch.unique(y[i]) for i in range(bs)]

[TensorMask([0, 3]), TensorMask([0, 4]), TensorMask([0, 3]), TensorMask([0])]
{% endraw %}

Pure Pytorch

The fast.ai Datablock API is very useful, we can load the images and apply transforms very quickly. On the other side it can be a limit when experimenting with other models or with custom transformations and requires a deeper knoledge of the high-level API.

Here some functions and classes from this kernel for pure Pytorch Dataset and DataLoader.

{% raw %}
{% endraw %} {% raw %}

get_train_dls[source]

get_train_dls(phase, mean=None, std=None, batch_size=8, num_workers=4)

Returns dataloader for the model training.

{% endraw %} {% raw %} 
steel_dls = get_train_dls(phase='train')
test_eq(len(steel_dls), 1257)
{% endraw %} {% raw %} 
xb, yb = next(iter(steel_dls))
xb.shape, yb.shape

(torch.Size([8, 3, 256, 1600]), torch.Size([8, 4, 256, 1600]))
{% endraw %}